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LONG-TERM  GOALS 

The  long-tenn  goal  of  this  RTP  project  is  to  provide  the  warfighter  with  superior  battlespace 
environmental  awareness  in  terms  of  high  fidelity  four-dimensional  (4D)  depiction  of  the  global 
atmospheric  state.  This  situational  awareness  is  a  key  aspect  of  information  superiority  in  the  DoD’s 
strategic  plan  to  ensure  battlespace  dominance  in  the  21st  century.  This  goal  is  to  be  accomplished  by 
providing  NOGAPS 1 2  with  the  best  possible  set  of  initial  conditions  through  the  use  of  a  next 
generation  global  atmospheric  4D  variational  (4DVAR)  data  assimilation  system,  NAVDAS-AR  . 

OBJECTIVES 

The  objective  of  this  project  is  to  construct  and  transition  a  4DVAR  global  atmospheric  data 
assimilation  system  for  NOGAPS  to  the  Fleet  Numerical  Meteorology  and  Oceanography  Center 
(FNMOC).  This  system,  NAVDAS-AR,  represents  the  first  operational,  weak  constraint,  4DVAR 
atmospheric  data  assimilation  system  in  the  world.  In  this  context,  “weak  constraint”  means  that  the 
atmospheric  forecast  model  is  not  considered  a  “perfect”  model,  but  rather  is  assumed  to  have  errors. 
This  enables  the  most  optimal  solution.  NAVDAS-AR  will  provide  high  fidelity,  dynamically 
consistent  analyses  for  NWP  model  initialization  and  for  warfighter  support,  and  will  be  capable  of 
efficiently  handling  large  numbers  of  observations  that  may  be  irregularly  distributed  in  space  and 
time,  and/or  indirectly  related  to  the  model  state  variables  (e.g.,  satellite  radiances  or  wind  vectors). 

APPROACH 

Our  approach  is  to  build  on  the  prototype  of  NAVDAS-AR  (Xu  et  al  2005,  and  Rosmond  and  Xu 
2006)  that  has  been  developed  and  successfully  applied  to  the  global  4DVAR  data  assimilation 
application  using  the  NOGAPS  prediction  model  as  a  dynamic  constraint.  This  project,  which  is  a 
follow-up  to  a  NRL  ongoing  in-house  6.2  data  assimilation  project,  will  expand  this  prototype  to  a 
next-generation  operational  global  atmospheric  data  assimilation  system.  We  will  leverage  the 
existing  NAVDAS  and  NOGAPS  infrastructures  to  provide  the  pre-  and  post-  analysis  processes.  The 
system  will  be  thoroughly  tested  using  scientific  studies,  and  comprehensive  data  assimilation  and 
forecast  experiments.  Although  the  goals  are  ambitious,  they  are  realistic  because  the  theoretical  basis 
for  the  project  is  already  in  place  owing  to  great  progress  made  in  our  6.1  and  6.2  in-house  data 
assimilation  projects  on  variational  data  assimilation. 


1  NOGAPS:  Navy  Operational  Global  Atmospheric  Prediction  System 

2  NAVDAS-AR:  NRL  Atmospheric  Variational  Data  Assimilation  System  -  Accelerated  Representer 
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WORK  COMPLETED 


The  following  is  a  list  of  work  completed  related  to  this  project  during  FY06. 

1 .  Developed  and  tested  a  new  efficient  computational  algorithm  for  initial  error  covariance 
convolution. 

2.  Increased  the  scalability  of  NAVDAS-AR  to  0(100)  processors  in  distributed  computer 
architectures. 

3.  Developed  and  tested  a  new  stopping  algorithm  that  requires  fewer  iterations  to  achieve  the  same 
accuracy. 

4.  Completed  a  generalized  framework  of  observation  operators  for  NAVDAS-AR  and  started  to 
assimilate  AMSU-A  radiances. 

5.  Ported  the  system  to  the  NRL  Linux  cluster  and  HPC  computers  to  facilitate  subsequent  extensive 
data  assimilation  testing. 

These  FY06  accomplishments  are  critical  for  conducting  subsequent  comprehensive  data  assimilation 

tests  planned  for  FY07. 

RESULTS 

The  following  results  represent  some  highlights  of  the  several  significant  accomplishments  of  this 

project  during  FY06. 

A  new  stopping  algorithm  for  NA  VDAS-AR 


Figure  1:  Total  number  of  iterations  required  for  new  and  current  stopping  algorithms  in 
NA  VDAS-AR  [Graphic  shows  the  new  stopping  algorithm  requires  only  half  of  the  computation 
(number  of  total  iterations)  to  achieve  the  same  accuracy] 
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An  outer-inner  loop  strategy  is  used  in  NAVDAS-AR.  While  the  outer  loop  deals  with  the 
nonlinearities  that  commonly  exist  in  global  atmospheric  processes,  the  inner  loop  deals  strictly  with 
the  linearized  process  related  to  the  minimization  along  a  previous  basic  state.  An  iterative  conjugate 
gradient  solver  is  used  in  the  inner  loop  to  minimize  a  four  dimensional  cost  function  that  represents 
the  misfit  between  the  initial  condition,  the  model,  the  observations  and  truth,  respectively.  The  total 
computational  cost  of  NAVDAS-AR  approximately  equals  the  total  number  of  inner  loop  iterations 
required  from  all  the  outer  loops.  Typically,  we  should  get  the  4DVAR  solution  as  accurate  as 
possible  in  the  first  outer  loop,  such  that  we  are  able  to  capture  the  majority  of  the  useful  information 
in  the  first  outer  loop.  The  original  version  of  NAVDAS-AR  reflects  this  practice.  However,  we 
recently  found  that  it  is  much  more  computationally  efficient  to  relax  the  stopping  criteria  throughout 
the  whole  data  assimilation  process  (including  the  first  outer  loop)  to  achieve  the  same  analysis 
accuracy.  The  advantage  of  the  new  stopping  algorithm  is  clearly  demonstrated  in  Figure  1 . 

Recent  improvements  to  NA  VDAS-AR 

Recent  improvements  to  NAVDAS-AR  have  been  very  dramatic.  The  most  computationally 
expensive  part  of  the  original  NAVDAS-AR  is  the  convolution  of  the  initial  background  error 
covariance  with  the  initial  state  vector.  There  were  two  main  issues  associated  with  cost  in  the  original 
version,  namely  the  MPI  scalability  and  the  background  error  covariance  calculations,  where  matrix 
operations  dominate.  We  have  made  significant  gains  (particularly  on  the  IBM  architecture 
computers),  not  only  in  the  NAVDAS-AR  itself,  but  also  the  NOGAPS  forecast  model.  The  following 
are  comparisons  of  typical  speedups  for  NAVDAS-AR  (the  “sweep”  algorithm)  and  the  NOGAPS 
forecast  model. 

Table  1:  Computational  requirement  in  seconds  for  original  and  new  NA  VDAS-AR  and  NOGAPS, 

respectively  [Table  shows  that  the  new  AR  and  NOGAPS  require  less  than  60%  and  about  70%  of 
the  computation  required  by  the  original  AR  and  NOGAPS,  respectively.] 


NAVDAS-AR  (50  iterations) 

NOGAPS  (120  hr  forecast) 

Original 

1620  secs 

4540  secs 

New 

940  secs 

3244  secs 

In  both  cases  the  most  dramatic  speed  decreases  come  from  consistent  use  of  math  libraries  that  are 
optimized  for  the  platform  we  are  running  on,  while  maintaining  code  portability  between  different 
platforms). 

In  NAVDAS-AR,  it  is  also  necessary  to  switch  between  lat-lon  'space'  and  AR  volume  (prism)  'space' 
in  the  background  covariance  convolution.  In  the  past,  this  was  been  done  in  a  variety  of  ways:  the 
late  Dr.  Roger  Daley  used  a  'master-slave'  MPI  algorithm  lifted  directly  from  NAVDAS  ;  we  switched 
this  to  collective  MPI  calls  to  exploit  the  static  grid  of  the  AR  covariance  calculations.  Both  of  these 
approaches  scale  very  poorly,  but  Dr.  Tom  Rosmond  recently  developed  a  point-to-point  MPI 
algorithm  that  scale  well.  The  cost  of  the  lat-lon  to  AR  volume  transfers  are  much  less  than  the  AR 
volume  to  lat-lon  transfers.  Fortunately,  our  new  method  applies  to  the  latter,  so  the  benefits  are  great. 
On  40  processors  the  new  method  is  typically  5  times  faster  than  the  old  (100  secs  vs.  500  secs),  so 


3  NAVDAS:  NRL  Atmospheric  Variational  Data  Assimilation  System 
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about  50%  of  the  total  AR  speedup  is  due  to  this  change,  the  other  50%  from  the  switch  to  the  platform 
specific  libraries. 

In  summary,  these  changes  will  have  a  dramatic  impact  on  our  throughput  for  AR  experiments.  A  full 
AR  update  cycle  on  the  FNMOC  IBM,  including  background  forecast,  QC,  AR  solution,  SLP  analysis, 
post-processing,  graphics,  and  statistics,  takes  about  30  minutes.  The  operational  NAVDAS,  which  is 
run  on  the  FNMOC  SGI  machines,  also  takes  around  30  minutes.  The  recent  performance 
enhancements  make  employing  higher  inner  loop  resolution  more  feasible,  which  is  desirable,  as  this 
may  be  necessary  for  special  cases,  such  as  the  assimilation  of  data  associated  with  tropical  cyclones. 


NOGAPS  500  MB  anomaly  correlation  using  NA  VADS-AR 


From  a  theoretical  point  of  view,  NAVDAS-AR  is  superior  to  NAVDAS,  in  part  because  NOGAPS  is 
used  as  a  dynamical  constraint  and  observations  are  assimilated  at  the  correct  observation  time.  It  is 
equally  important  that  we  implement  the  “AR”  algorithm  properly,  such  that  the  theoretical  superiority 
translates  into  the  real  gains  in  operational  forecasts.  In  the  past  several  months,  we  were  able  to 
directly  assimilate  satellite  radiances,  such  as  AMSU-A  into  NAVDAS-AR.  Consequently,  we  are 
now  able  to  conduct  comparisons  of  NOGAPS  forecast  skill  between  NAVDAS-AR  and  NAVDAS 
update  cycles.  An  important  benchmark  test  of  the  skill  of  the  data  assimilation  system  is  to  compare 
NOGAPS  500  MB  height  anomaly  correlations  associated  with  the  two  systems. 


NOGAPS  DATA  ASSIMILATION  TEST  NOGAPS  DATA  ASSIMILATION  TEST 


500  MB  NORTH  HEM  HEIGHT  ANOMALY  COR  500  MB  SOUTH  HEM  HEIGHT  ANOMALY  COR 


2006021500  -  2006030700  2006021500  -  2006030700 


TOM-AR  - OPS 


TOM-AR  - OPS 


Figure  2:  500  MB  forecast  height  anomaly  correlations  out  to  5  days  (120  hrs)  for  the  a)  Northern 
Hemisphere,  and  b)  Southern  Hemisphere.  The  solid  red  line  is  for  NOGAPS  forecasts  initialized 
NA  VDAS-AR  analyses,  while  the  solid  black  line  is  for  NOGAPS  forecasts  initialized  with 

NA  VDAS  analyses. 


Figure  2  shows  the  statistic  of  NOGAPS  500  MB  Northern  and  Southern  Hemisphere  height  anomaly 
correlations,  respectively,  for  up  to  120  hr  global  forecasts.  The  solid  red  lines  represent  the  results 
using  NAVDAS-AR,  while  the  solid  black  lines  represent  the  results  using  NAVDAS.  The  statistics 
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were  accumulated  during  a  three- week  time  period  between  00Z  15  Feb  2006  and  00Z  07  March  2006. 
The  operational  run  (using  NAVDAS)  has  more  data,  but  an  optimized,  operational  system, 
NAVDAS-AR  would  benefit  from  more  observations,  but  suffers  from  lack  of  optimization,  and  a 
known  bias  issue.  As  indicated  in  Figure  2,  the  red  and  black  lines  virtually  overlap  each  other. 
Despite  these  discrepancies  in  the  data  used  in  each  of  the  runs,  the  results  suggest  that  the  NAVDAS- 
AR  forecast  skill,  after  the  first  year  of  the  RTP  effort,  is  essentially  equivalent  to  the  operational 
NAVDAS.  These  results  are  very  encouraging,  as  the  observation  selection  and  quality  control  have 
not  yet  been  optimized  for  NAVDAS-AR,  and  the  system  has  not  been  fine-tuned.  Also,  NAVDAS- 
AR  will  be  a  perfect  candidate  to  handle  the  huge  volume  of  current  and  future  observations,  such  as 
Atmospheric  infrared  Sounder  (AIRS),  that  we  are  planning  to  assimilate  in  the  future. 

IMPACT/APPLICATIONS 

The  current  operational  data  assimilation  system  at  FNMOC,  NAVDAS,  is  based  on  a  three- 
dimensional  variational  (3DVAR)  algorithm  and  is  cast  in  observation  space.  The  3DVAR  algorithm 
is  widely  used  in  intermittent  cycling  data  assimilation  for  the  analysis  of  global  and  synoptic  scales 
around  the  world.  It  can  handle  relatively  slowly  evolving  flows  and  observation  platforms  that 
sample  heterogeneously  in  space,  but  assumes  that  the  observations  are  taken  at  the  analysis  time. 
However,  highly  intennittent  flows  that  are  not  governed  by  simple  balance  relationships,  and 
observation  systems  that  sample  irregularly  in  time,  or  with  high  temporal  frequency,  are  not  well 
accommodated  within  an  intennittent  3DVAR  framework  but  can  be  accommodated  by  a  4D  data 
assimilation  system.  Furthennore,  an  intermittent  3DVAR  algorithm  produces  a  “snapshot”  of  the 
atmosphere  at  the  center  of  the  typical  6-hour  observation  time  window,  automatically  making  the 
resulting  atmospheric  analysis  at  least  3  hours  old. 

With  NAVDAS-AR,  a  continuous  picture  of  the  atmosphere  over  the  observation  time  window  is 
produced,  providing  an  atmospheric  analysis  at  the  end  of  the  time  window  that  is  current  rather  than  3 
hours  old.  Although  NAVDAS  has  been  quite  successful,  a  4D  data  assimilation  system  is  a  necessity 
to  significantly  improve  not  only  the  accuracy  of  the  common  operational  picture  required  by  the 
warfighter  but  also  the  timeliness  of  providing  this  more  accurate  picture  to  the  warfighter.  The 
advanced  4DVAR  data  assimilation  algorithm,  NAVDAS-AR,  will  provide  the  basis  for  this  system. 
Only  through  4DVAR  algorithms  can  we  truly  exploit  many  of  the  observations  from  current  and 
future  observing  systems.  This  is  especially  important  for  remotely  sensed  observations  that  are 
nonlinearly  and  indirectly  related  to  the  model  state  variables  (e.g.,  satellite  radiances  and  GPS  radio 
occultation  measurements).  In  addition,  the  computational  efficiency  of  NAVDAS-AR  with  respect  to 
the  number  of  observations  makes  it  more  efficient  than  the  NAVDAS  3DVAR  system  in  handling  the 
monumental  increase  in  the  volume  of  satellite  data  expected  over  the  next  decade. 

TRANSITIONS 

Improved  algorithms  for  NAVDAS-AR  have  been  transitioned  to  the  6.4  component  of  this  project, 
and  will  ultimately  be  transitioned  to  FNMOC  as  the  Navy’s  next  generation  operational  global 
atmospheric  data  assimilation  system. 
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RELATED  PROJECTS 


Some  of  the  technologies  developed  for  this  project  will  be  used  immediately  to  improve  the  current 

operational  data  assimilation  system  and  the  observational  impact  in  other  NRL  projects.  NAVDAS- 

AR  has  been  recognized  as  a  viable  framework  for  various  data  assimilation  applications,  and  this  is 

now  being  started  in  an  NRL  ocean  data  assimilation  system  project. 
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